Skip to content

Conversation

@Ceng23333
Copy link
Collaborator

add memory usage stats from allocator && mem ref count, mem pool implementation

#513

# Memory Statistics Functions
# ==============================================================================

def print_memory_stats(title="Memory Statistics", show_detailed=False):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个函数和matmul有关吗,为什么会在matmul的测试脚本里

thread_local Runtime *ContextImpl::current_runtime_ = nullptr;

Runtime *ContextImpl::getCurrentRuntime() {
if (current_runtime_ == nullptr) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

context的构造器应该已经保证了一定有建好的runtime,什么时候get current runtime会是nullptr?

Copy link
Collaborator Author

@Ceng23333 Ceng23333 Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这种闭包用法会出现current runtime未初始化
`

    const int allocations_per_thread = 100;
    std::vector<std::thread> threads;
    std::atomic<int> success_count{0};
    std::atomic<int> failure_count{0};
    for (int i = 0; i < num_threads_; ++i) {
        threads.emplace_back([&, i]() {
            try {
                for (int j = 0; j < allocations_per_thread; ++j) {
                    // Allocate memory of random size
                    size_t size = 64 + (j % 1024);
                    spdlog::debug("Thread {}: ConcurrentAllocations: Allocating memory of size {}", i, size);
                    auto memory = context::allocateMemory(size);
                    spdlog::debug("Thread {}: ConcurrentAllocations: Memory allocated successfully", i);
                    if (memory && memory->size() == size) {
                        success_count++;
                    } else {
                        failure_count++;
                    }

                    // Small delay to increase chance of race conditions
                    std::this_thread::sleep_for(std::chrono::microseconds(1));
                }
            } catch (const std::exception &e) {
                failure_count++;
                std::cerr << "Thread " << i << " failed: " << e.what() << std::endl;
            }
        });
    }

`

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

明白了,创建新的线程的时候会是nullptr。这是个bug,要不单独提pr修复一下吧

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK,这部分我先单独提pr

: ptr(p), size(s), ref_count(1), is_freed(false) {}
};

class MemoryPool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include目录下都是会暴露给外部的接口,这个设计是要把memory pool的接口暴露给外部吗?什么情况下用户或者上层框架需要直接和memory pool交互?

stats_.allocation[static_cast<size_t>(StatType::AGGREGATE)].increase(1);
stats_.requested_bytes[static_cast<size_t>(StatType::AGGREGATE)].increase(size);

INFINICORE_CHECK_ERROR(infinirtMallocAsync(&ptr, size, context::getStream()));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你这不还是用的原生的mallocAsync接口,那你的内存池用哪了?


namespace infinicore {

MemoryPool &MemoryPool::instance() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为什么memory pool是 singleton的设计

@PanZezhong1725 PanZezhong1725 marked this pull request as draft October 21, 2025 02:02
@Ceng23333 Ceng23333 changed the title issue/513: 内存使用情况统计&&内存引用计数和内存池实现 issue/513: 内存使用情况统计&&内存引用计数和内存池实现(WIP) Oct 21, 2025
Signed-off-by: Ceng23333 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants